Emergence of Cooperation on Complex Networks

Department of PhysicsWe studied the structural determinants of evolutionary dynamics on complex networks in the paradigm of social dilemma games and mathematical graph theory. Motivated by the renewed interest in the evolution of altruistic behavior in the population of rational individuals, who strive to maximize their own benefit, we qunatitatively explored the impact of the various structural characteristics on the fixation of a single cooperator in the sea of defectors. In non-complete graphs, where an individual???s fitness is determined not only by the individual???s strategy but also by its local neighbors??? strategies, the fixation of the mutant species significantly deviates from the prediction of the traditional Moran process occurring in a wellmixed population. From the numerical experiments and analytical approximations on the ensembles of populations prescribed by a suite of structural characteristics, we map the region of facilitated altruism in the space of degree heterogeneity, clustering properties, and degree-degree correlations. Of particular interest are the attributes of the seed cooperator that leads to the fixation of cooperators with higher chance. We show that, though the degree of the seed cooperator can either positively or negatively impact the cooperation depending on the numeric details of the payoff matrix, the low average degree, degree homogeneity, and the negative degree-degree correlation is always favoured for the evolution of the social behavior. To explain these findings in a more intuitive manner, we introduced the concept of the structural reciprocity in the framework of evolutionary game theory and propose possible structural intervention strategies to promote the altruism with no centralized control. To avoid hasty generalization at all costs, however, it should be noted that the underlying population topology is not the only determinant of the fixation of cooperators. Yet another determinants are the details of the update rules and fitness-payoff relations. The interplay between these dynamical rules and the structural characteristics should be a fruitful avenue for future research.clos

Hydra: Multi-head Low-rank Adaptation for Parameter Efficient Fine-tuning

The recent surge in large-scale foundation models has spurred the development of efficient methods for adapting these models to various downstream tasks. Low-rank adaptation methods, such as LoRA, have gained significant attention due to their outstanding parameter efficiency and no additional inference latency. This paper investigates a more general form of adapter module based on the analysis that parallel and sequential adaptation branches learn novel and general features during fine-tuning, respectively. The proposed method, named Hydra, due to its multi-head computational branches, combines parallel and sequential branch to integrate capabilities, which is more expressive than existing single branch methods and enables the exploration of a broader range of optimal points in the fine-tuning process. In addition, the proposed adaptation method explicitly leverages the pre-trained weights by performing a linear combination of the pre-trained features. It allows the learned features to have better generalization performance across diverse downstream tasks. Furthermore, we perform a comprehensive analysis of the characteristics of each adaptation branch with empirical evidence. Through an extensive range of experiments, encompassing comparisons and ablation studies, we substantiate the efficiency and demonstrate the superior performance of Hydra. This comprehensive evaluation underscores the potential impact and effectiveness of Hydra in a variety of applications. Our code is available on \url{https://github.com/extremebird/Hydra

Separable Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) have recently emerged as promising data-driven PDE solvers showing encouraging results on various PDEs. However, there is a fundamental limitation of training PINNs to solve multi-dimensional PDEs and approximate highly complex solution functions. The number of training points (collocation points) required on these challenging PDEs grows substantially, but it is severely limited due to the expensive computational costs and heavy memory overhead. To overcome this issue, we propose a network architecture and training algorithm for PINNs. The proposed method, separable PINN (SPINN), operates on a per-axis basis to significantly reduce the number of network propagations in multi-dimensional PDEs unlike point-wise processing in conventional PINNs. We also propose using forward-mode automatic differentiation to reduce the computational cost of computing PDE residuals, enabling a large number of collocation points (>10^7) on a single commodity GPU. The experimental results show drastically reduced computational costs (62x in wall-clock time, 1,394x in FLOPs given the same number of collocation points) in multi-dimensional PDEs while achieving better accuracy. Furthermore, we present that SPINN can solve a chaotic (2+1)-d Navier-Stokes equation significantly faster than the best-performing prior method (9 minutes vs 10 hours in a single GPU), maintaining accuracy. Finally, we showcase that SPINN can accurately obtain the solution of a highly nonlinear and multi-dimensional PDE, a (3+1)-d Navier-Stokes equation.Comment: arXiv admin note: text overlap with arXiv:2211.0876

Separable PINN: Mitigating the Curse of Dimensionality in Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) have emerged as new data-driven PDE solvers for both forward and inverse problems. While promising, the expensive computational costs to obtain solutions often restrict their broader applicability. We demonstrate that the computations in automatic differentiation (AD) can be significantly reduced by leveraging forward-mode AD when training PINN. However, a naive application of forward-mode AD to conventional PINNs results in higher computation, losing its practical benefit. Therefore, we propose a network architecture, called separable PINN (SPINN), which can facilitate forward-mode AD for more efficient computation. SPINN operates on a per-axis basis instead of point-wise processing in conventional PINNs, decreasing the number of network forward passes. Besides, while the computation and memory costs of standard PINNs grow exponentially along with the grid resolution, that of our model is remarkably less susceptible, mitigating the curse of dimensionality. We demonstrate the effectiveness of our model in various PDE systems by significantly reducing the training run-time while achieving comparable accuracy. Project page: https://jwcho5576.github.io/spinn/Comment: To appear in NeurIPS 2022 Workshop on The Symbiosis of Deep Learning and Differential Equations (DLDE) - II, 12 pages, 5 figures, full paper: arXiv:2306.1596

Quantitative Screening of Cervical Cancers for Low-Resource Settings: Pilot Study of Smartphone-Based Endoscopic Visual Inspection After Acetic Acid Using Machine Learning Techniques

Background: Approximately 90% of global cervical cancer (CC) is mostly found in low- and middle-income countries. In most cases, CC can be detected early through routine screening programs, including a cytology-based test. However, it is logistically difficult to offer this program in low-resource settings due to limited resources and infrastructure, and few trained experts. A visual inspection following the application of acetic acid (VIA) has been widely promoted and is routinely recommended as a viable form of CC screening in resource-constrained countries. Digital images of the cervix have been acquired during VIA procedure with better quality assurance and visualization, leading to higher diagnostic accuracy and reduction of the variability of detection rate. However, a colposcope is bulky, expensive, electricity-dependent, and needs routine maintenance, and to confirm the grade of abnormality through its images, a specialist must be present. Recently, smartphone-based imaging systems have made a significant impact on the practice of medicine by offering a cost-effective, rapid, and noninvasive method of evaluation. Furthermore, computer-aided analyses, including image processing-based methods and machine learning techniques, have also shown great potential for a high impact on medicinal evaluations

Encoder-Weighted W-Net for Unsupervised Segmentation of Cervix Region in Colposcopy Images

Simple Summary The cervix region segmentation significantly affects the accuracy of diagnosis when analyzing colposcopy. Detecting the cervix region requires manual, intensive, and time-consuming labor from a trained gynecologist. In this paper, we propose a deep learning-based automatic cervix region segmentation method that enables the extraction of the region of interest from colposcopy images in an unsupervised manner. The segmentation performance with a Dice coefficient improved from 0.612 to 0.710 by applying the proposed loss function and encoder-weighted learning scheme and showed the best performance among all the compared methods. The automatically detected cervix region can improve the performance of image-based interpretation and diagnosis by suggesting where the clinicians should focus during colposcopy analysis. Cervical cancer can be prevented and treated better if it is diagnosed early. Colposcopy, a way of clinically looking at the cervix region, is an efficient method for cervical cancer screening and its early detection. The cervix region segmentation significantly affects the performance of computer-aided diagnostics using a colposcopy, particularly cervical intraepithelial neoplasia (CIN) classification. However, there are few studies of cervix segmentation in colposcopy, and no studies of fully unsupervised cervix region detection without image pre- and post-processing. In this study, we propose a deep learning-based unsupervised method to identify cervix regions without pre- and post-processing. A new loss function and a novel scheduling scheme for the baseline W-Net are proposed for fully unsupervised cervix region segmentation in colposcopy. The experimental results showed that the proposed method achieved the best performance in the cervix segmentation with a Dice coefficient of 0.71 with less computational cost. The proposed method produced cervix segmentation masks with more reduction in outliers and can be applied before CIN detection or other diagnoses to improve diagnostic performance. Our results demonstrate that the proposed method not only assists medical specialists in diagnosis in practical situations but also shows the potential of an unsupervised segmentation approach in colposcopy

Deep-Learning-Based Algorithm for the Removal of Electromagnetic Interference Noise in Photoacoustic Endoscopic Image Processing

Despite all the expectations for photoacoustic endoscopy (PAE), there are still several technical issues that must be resolved before the technique can be successfully translated into clinics. Among these, electromagnetic interference (EMI) noise, in addition to the limited signal-to-noise ratio (SNR), have hindered the rapid development of related technologies. Unlike endoscopic ultrasound, in which the SNR can be increased by simply applying a higher pulsing voltage, there is a fundamental limitation in leveraging the SNR of PAE signals because they are mostly determined by the optical pulse energy applied, which must be within the safety limits. Moreover, a typical PAE hardware situation requires a wide separation between the ultrasonic sensor and the amplifier, meaning that it is not easy to build an ideal PAE system that would be unaffected by EMI noise. With the intention of expediting the progress of related research, in this study, we investigated the feasibility of deep-learning-based EMI noise removal involved in PAE image processing. In particular, we selected four fully convolutional neural network architectures, U-Net, Segnet, FCN-16s, and FCN-8s, and observed that a modified U-Net architecture outperformed the other architectures in the EMI noise removal. Classical filter methods were also compared to confirm the superiority of the deep-learning-based approach. Still, it was by the U-Net architecture that we were able to successfully produce a denoised 3D vasculature map that could even depict the mesh-like capillary networks distributed in the wall of a rat colorectum. As the development of a low-cost laser diode or LED-based photoacoustic tomography (PAT) system is now emerging as one of the important topics in PAT, we expect that the presented AI strategy for the removal of EMI noise could be broadly applicable to many areas of PAT, in which the ability to apply a hardware-based prevention method is limited and thus EMI noise appears more prominently due to poor SNR

Emergence of Altruism in Spatially Structured Populations

Spatial structure of an otherwise random and uniform population impacts on the emergence of cooperation among selfish and rational individuals. In the ensembles of structured populations constrained by the combinations of degree distribution, assortativity, and clustering coefficient, we try to map the comprehensive ???cooperativity landscape??? in the context of game theory. To explore the structure space with varying degrees of assortativity and higher-order degree-degree correlations, we employ a Monte Carlo algorithm for the degree-preserving rewiring and produce the statistical ensemble of thermally annealed graphs retaining the major characteristics of empirical social networks. We find that the effects of the clustering on the sustained cooperation are not monotonic whereas the higher assortativity is disfavored in general. We discuss the individual-based reinforcement strategies for the enhanced cooperativity in a variety of real-world social networks

Selection of Cooperation in Spatially Structured Populations

The social dilemma games give rise to an emergence of cooperation in which altruistic individuals survive the natural selection at higher rate than random chance. We try to extend our understanding of this spatial reciprocity by including the impact of degree-degree correlation on the propensity toward prosocial behaviour in an otherwise well-mixed population. In a stochastic death-birth process with weak selection, we find that the disassortative degree mixing, or negative correlation between the degrees of neighbouring nodes significantly promotes the fixation of cooperators whereas the assortative mixing acts to suppress it. This is consistent with the fact that the spatial heterogeneity weakens the average tendency of a population to cooperate, which we describe in a unified scheme of the effective isothermality in coarse-grained networks. We also discuss the individual-level incentives that indirectly foster restructuring the social networks toward the more cooperative topologies